1. Field of the Invention
The present invention relates to a technique of increasing the resolution of a frame of a low-resolution moving image using a super-resolution processing technique or the like.
2. Description of the Related Art
Some recent image capturing apparatuses such as digital cameras and digital video cameras have a function of capturing a moving image of high frame rate using a high shutter speed. The technique of capturing a moving image of high frame rate is utilized in a scene where the object may be blurred or a scene where the opportunity to capture the image is too important to miss. When capturing and recording such a moving image of high frame rate, the number of frames captured in one second is larger than in normal image capture. Hence, the number of pixels must be decreased in a moving image of higher frame rate in accordance with the limits of processing capability for the recording process. That is, in the moving image of high frame rate, the resolution of one frame image is low. This is because if raising the frame rate in image capture may make the read speed of an image sensor such as a CMOS sensor or the write speed in a recording medium exceed the limit, the recording process may fail.
On the other hand, display resolutions of display devices such as TV sets are increasing, considering the start of digital TV broadcasting. For this reason, when the display device with the high display resolution displays the above-described low-resolution moving image of high frame rate while enlarging it based on the display area, the viewer has an impression that the image lacks sharpness. Especially, displaying a frame image as a still image by, for example, making the moving image pause gives the viewer an impression that the image lacks sharpness.
There is known a technique of, when, for example, making a moving image of high frame rate pause, generating a high-resolution image by referring to a plurality of low-resolution frame images and applying a super-resolution process, and displaying the image on a display device. Japanese Patent Laid-Open No. 2005-197910 discloses a technique of determining, based on the ratio of the display resolution to the resolution of a frame of a moving image of high frame rate, the number of frames to be referred to in the super-resolution process.
In the super-resolution process, generally, as the number of low-resolution images to be referred to increases, the quality of an obtained higher-resolution image improves. In addition, unless low-resolution images to be referred to contain position difference of a moving object or a change in the light source to some extent, it is difficult to generate interpolated pixels between pixels of a higher-resolution image. For these reasons, if the number of frames to be referred to in the super-resolution process is determined based on only the ratio of the display resolution to the resolution of a moving image, as in the prior art, it may be impossible to obtain a high-quality result when, for example, the frame rate is high, and position difference of an object rarely exists between the frames.